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Software  Acquisition  Resource 
Expenditure  (SARE)  Reporting 


Part  1 
Part  2 

Part  3 
Part  4 
Part  5 


—  Current  financial  data  collection  method 

—  The  SAKE  reporting  methodology  and 
contracting  for  SARE  reporting 

—  Project  historical  file  contents 

-  Data  usage  for  software  development  monitoring 

—  Costs,  questions, and  future  multiproject  data 
base 


■y 

SARE  reporting  is  a  data  collection  methodology  used  to  collect 
software-unique  financial  data  plu9  technical  data  that  make  the 
financial  data  meaningful.  The  data  can  be  used  to  monitor  the 
progress  of  software  development  work  on  the  contract  in  which  the 
data  is  collected.  Also,  the  data  is  to  be  submitted  to  a 
multiproject  Air  Force  Systems  Command  (AFSC)  software  data  base. 
The  data  base  will  help  formulate,  calibrate,  and  validate  software 
cost/schedule  estimation  methods. 

The  discussion  of  SARE  reporting  consists  of  five  parts: 

1.  A  brief  review  of  the  current  AFSC  financial  data 
collection  methods 

2.  A  discussion  of  the  SARE  reporting  methodology  and 
comments  on  how  the  Government  contracts  for  SARE  reporting 

3.  An  overview  of  software  data  reported  and  inserted 
into  a  project  historical  file 

4.  The  use  of  the  data  collected  during  a  contract  to 
monitor  software  development  during  that  contract 

5.  A  discussion  of  the  methodology  features  questioned 
by  reviewers,  and  the  future  use  of  the  software  data  in  a 
multiproject  data  base 
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Current  Method - C/SCSC 


Data  Reported  at  Level  3  of 


Typically,  standard  cost  performance  data  is  collected  by  the 
AFSC  tor  level  3  contract  work  breakdown  structure  (CWBS)  elements 
under  the  terms  of  Data  Item-F-6000C»  This  data  item  is  consistent 
with  the  Cost/Schedule  Control  System  Criteria  (C/SCSC)  of  DOD 
Instruction  7000.2.  The  data  is  reported  on  computer  cards,  is  read 
by  a  computer,  and  can  be  printed  out  by  the  computer  for  system 
program  office  (SPO)  use.  The  software  package  that  processes  the 
data  is  called  the  "Automated  Financial  Analysis  Program."  It  is 
maintained  and  operated  by  the  Cost  Analysis  Division,  Comptroller, 
Electronic  Systems  Division,  AFSC  at  the  Hanscom  Air  Force  Base. 

Since  software  costs  are  lumped  together  in  level  3  CWBS 
elements,  poor  software  performance  may  not  be  revealed  by  level  3 
CWBS  financial  data  collected  under  the  standard  cost  performance 
report.  For  example,  a  problem  revealed  by  financial  data  of  one 
Computer  Program  Configuration  Item  (CPCI)  can  be  masked  by  the 
financial  data  of  a  second  CPCI.  In  addition,  well-structured 
technical  date  is  not  collected  under  „he  standard  cost  performance 
report.  Technical  data  is  needed  to  identify  and  define,  software 
components  associated  with  financial  data  indicating  procurement 
problems;  and  to  explain  the  problems  (e.g.,  an  unexpected  large 
increase  in  the  size  of  a  software  component). 
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SARE  Reporting  —  A 
Methodology 


»  I  t 


As  stated  in  the  introduction  to  the  briefing,  SARE  reporting 
'is  a  data  collection  methodology.  It  is  used  to  collect  software- 
unique  financial  data  plus  technical  data  that  make  the  financial 
data  meaningful.  In  practice,  this  methodology  includes: 

1.  Inclusion  of  standard,  well-defined,  software-related 
elements  in  the  CWBS,  in  sufficient  detail  to  account  for  all 
software  procurement  costs 

2.  Contractor  collection  of  financial  data  against  the  lowest 
level,  standard,  software-related  CWBS  elements 

3.  Contractor  collection  of  technical  data  that  characterize 
developed  computer  programs,  personnel  responsible  for  the 
development,  and  the  development  process 

4.  Contractor  submission  to  the  Government  of  software-related 
financial  and  technical  data  on  magnetic  tape 

3.  Government  computer  programs  reading  the  financial  and 
technical  data  off  the  magnetic  tapes  and  distributing  the  data  into 
a  pro  ject- unique  historical  file 

6.  Government-automated  generation  of  well-structured 
printouts,  outputting  data  selected  from  the  historical  file 
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Components  Used  to 
Contract  for  SARE  Reporting 


To  contract  for  the  SARE  reporting  methodology,  one  must 
Include; 

1.  Level  4  and  5  software-related  work  breakdown  structure 
(WBS)  elements  In  the  CWBS 

2.  The  requirement  of  RARE  reporting  lti  the  statement  of  work 

(SOW) 

3.  The  contract  data  requirements  list  (CDRL)  specification 
that  data  be  reported  on  magnetic  tape  according  to  the 
requirements  of  the  Software  Cost  Performance  Report  (S/W  CPR)  data 
item 

4.  SOW  references  to  MIL-STD-X  to  specify  the  definitions  of 
the  CWBS  elements 

5.  SOW  references  to  MIl.-STD-Y  to  specify  the  technical  data 
reporting  requirements 
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Project  Historical  File  Financial  Subreports 


NINE  SELECTABLE 
SUBREPORTS 


The  software  cost  performance  data  reported  each  month  on 
magnetic  tape  is  validated,  corrected  if  necessary,  and  inserted 
into  a  project  historical  file.  The  historical  file  consists  of 
nine  separate  subreports.  The  data  content  of  each  subreport 
accumulates  over  the  life  of  the  system  acquisition.  Four  of  the 
aubreport  types  are  categorized  as  financial.  They  include: 

1.  A  contract  identification  subreport 

2.  A  CWBS  subreport 

3.  A  contractor  organization  structure  subreport 

4.  A  cost  and  schedule  financial  data  subreport 

The  subreport  initials  shown  in  the  illustration  are  ID's  used  on 
the  magnetic  tape  to  identify  the  individual  subreports 

Data  is  collected  to  identify  the  prime  contractor  and  any 
software  subcontractors.  This  data  is  stored  in  Subreport  CID.  Also, 
data  that  defines  the  organization  structure  of  the  prime  contractor 
is  collected  and  stored  in  Subreport  COS. 
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Work  Breakdown  Structure  Subreport 


Data  is  collected  to  define  the  CWBS  and  1b  stored  in  Subreport 
WBS.  The  CWBS  encompasses  software  activities  associated  with: 

1.  Computer  program  configuration  item  (CPCI)  procurement 

2.  The  system-level,  software-related  work  associated  with 
CPCI' 8  working  together  in  an  integrated  system 

3.  Computer  program  component  (CPC)  design  and  coding,  where  a 
CPC  is  a  functionally  or  logically-distinct  part  of  a  CPCI 

MIL-STD-X  defines  a  standard  set  of  WBS  elements  used  in  the  CWBS. 
The  WBS  subreport  includes  all  CWBS  elements  for  levels  1  to  3,  and 
software-related  CWBS  elements  for  levels  4  to  6. 
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Cost  And  Schedule  Financial  Data 
Subreport 


MONTH 


Cost/schedule  financial  data  is  collected  each  month  and  stored 
in  the  historical  file  under  Subreport  CSD.  This  data  includes  for 
each  lowest  level  CWBS  element: 

I*  Budgeted  Cost  of  Work  Scheduled  (BCWS) 

2.  Budgeted  Cost  of  Work  Performed  (BCWP) 

3.  Actual  Cost  of  Work  Performed  (ACWP) 

A.  Budget  at  Completion  (BAC) 

5.  Latest  Revised  Estimate  at  Completion  (LRE) 

In  addition,  the  data  includes  actual  man-hours,  budgeted  man-hours, 

and  the  latest  revised  estimate  of  total  man-hours  at  completion. 

This  data  is  defined  by  the  C/SCSC  in  DOD  Instruction  7000*2  and  is  « 

required  by  DI-F-6000C,  the  standard  cost  performance  report.  j 

} 
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Project  Historical  File  Technical 
Subreports 


IUMWoJtS  SuVaEPORTS 


Five  of  the  project  historical  file  subreport  types  are 
categorised  as  technical.  They  include: 

1.  A  software  structure  subreport 

2.  A  schedule  subreport 

3.  A  software  components  characteristics  subreport 

4.  A  software  development  personnel  subreport 

S«  A  change  and  deviation  subreport 

The  data  required  by  each  subreport  is  defined  in  MIL-STD-Y. 


Software  Structure  Suhreport 


"VIMMU 


OPERATIONAL  FUNCTION 

— 

1.0 

Display 

a.o 

Avionics 

3.0 

4.0 

On-Line  Equipment  Diagnostic 

jSUPPORT  FUNCTION  1 

1.0 

Operating  System 

3.0 

Equipment  Maintenance 

3.0 

Software  Development 

4.0 

Off-Line  Data  Base  Management 

S.O 

Design 

A.O 

Teat  Software 

Data  la  collected  to  define  each  CPCI  to  be  procured  and  la 
stored  In  the  historical  file  under  Subreport  SST.  For  each  CPCI, 
the  functions  implemented  -must  be  selected  from  a  table.  The  table 
Index  of  each  selected  function  must  be  reported  on  magnetic  tape. 
For  example,  for  on-line  equipment  diagnostics,  one  or  more  of  the 
following  indexes  must  be  selected; 

1.  A, l  for  System  Readiness  Test 

2.  A, 2  for  Computer  Diagnostic 

3.  A. 3  for  Memory  Diagnostic 

A,  A. A  for  Display  Diagnostic 

5.  A. 5  for  Switch/Indicator  Panel  Diagnostics 

6.  A. 6  for  1/0  Diagnostic 

7.  A. 7  for  Mod*  Diagnostic 

8.  A. 8  for  Other 
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Software-Related  Schedule 
Subreport 


FINANCIAL  TECHNICAL 
SUBREPOFTS  3UBREP0RTS 


|CPCI  DEVELOPMENT  | 

1.1 

Design  Start 

1.2 

Code  Start 

1.3 

POR 

1.4 

CDR 

1.5 

Product  Spec.  Approval 

1.6 

Teet  Start 

1.7 

PQT 

1.6 

PQT 

1.9 

FCA 

1.10 

PCA 

1.11 

FOR 

t 


The  software  schedule  subreport  (Subreport  SRS)  provides 
estimates  of  CPCI  development  milestones  at  contract  award.  Also, 
the  actual  completion  date  of  each  event  is  reported  after  it 
occurs.  In  addition,  estimates  and  actual  values  associated  with 
system  development  test  and  evaluation  (DT&E)  procedures  start; 
system  DT&E  test  report  delivery;  installation  start;  and  program 
management  responsibility  transfer  are  reported  and  Inserted  into 
the  project  historical  file  under  Subreport  SRS. 


10 


VO-I7.W7 


Software  Components 
Characteristics  Subreport 


_ 

1.1 

Amount  of  Code 

1.2 

Code  per  Proceeding  Operation 

1.3 

Documentation 

(DIFFICULTY 

2.1 

Number  of  Inputs 

2.2 

Number  of  Outputs 

2.3 

Time  and  Storage  Criticality 

(development  methodology 

3.1 

Language  Used 

3.2 

Structure  and  Testing  Method 

3.3 

Code  Source 

3.4 

Design  Method 

The  software  components  characteristics  subreport  (Subreport 
SCS)  provide  estimates  of  the  size,  difficulty,  and  development 
methodology  at  contract  award.  At  the  end  of  the  contract,  actual 
values  are  reported  and  inserted  into  the  project  historical  file. 

As  an  example,  a  report  of  the  "Amount  of  Code"  for  a  CPCI  must 
include; 

1.  The  number  of  source  statements 

2.  The  number  of  source  statements  written  to  support  the  CPCI 
development. 

3.  The  number  of  object  program  words 
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The  software  development  personnel  subreport  (Subreport  SDP) 
provides  each  month  actual  counts  of  the  number  of  managers  and 
staff  personnel  working  on  the  project.  As  an  example  of  the  staff 
number,  each  monthly  S/W  CPR  magnetic  tape  must  Include: 

1.  The  number  of  staff  full-time  personnel 

2.  The  number  of  staff  personnel  new  to  the  project  since  the 
last  monthly  report 

3.  The  number  of  staff  part-time  personnel 
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Development  Changes  and 
Deviations  Subreport 


CHANGES _ 

1.0  Requirement*  Chans* 

2.0  D**lgn  Chang* 

3.0  Structure  Chang*  _ 

DEVIATIONS _ 

4.0  Documentation  not  prepared 
5.0  Review,  audit  or  teat  not 
completed 

6.0  Configuration  Management 
not  required _ 

ECPa  .  ~~ 

7.0  CPCI  ECPi 

6.0  Software-related  ayatem  level 


Data  ia  collected  under  Subreport  DCD  to  record  software 
development  changes  and  deviations,  Including: 

1.  Any  changes  in  the  original  plar 

2.  Any  deviations  from  common  practices  (e.g.,  any 
documentation  and  audits  of  CPCl's  waived) 

3.  Tt.e  number  of  engineering  change  proposals  (ECP's)  written 
against  each  CPCI 

A.  The  number  of  software-related,  system-level  ECP's 

For  example,  a  structure  change  could  be  a  CPCI  redefinition,  a  CPCI 
cancellation,  or  a  redefinition  of  the  CPC's  within  a  CPCI. 

Examples  of  deviations  Include  not  preparing  a  CPCI  development  or 
product  specification,  and  not  preparing  a  CPCI  test  procedure  or 
test  report. 
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Project  Historical  File  Financial 
Data  Evaluation 


The  project  historical  file  stores  software  data  collected  over 
the  life  of  the  system  acquisition.  This  data  can  be  processed  by 
Government  computers  to  detect  software  acquisition  problems.  Such 
problems  will  be  revealed  by  a  printout  listing  all  software 
activities  overrunning  budgeted  cost  or  behind  schedule. 

The  computer  program  that  implements  this  printout  will  check 
the  status  of  software  activity,  encompassed  by  each  CWBS  element; 
discard  the  elements  encompassing  activities  on  or  ahead  of 
schedule;  and  list  the  elements  associated  with  cost  overruns  and 
schedule  slips.  This  capability  significantly  reduces  the  amount  of 
paper  a  program  office  must  review.  In  fact,  a  one  or  two-page 
printout  should  encompass  all  software-related  activities  behind 
schedule  and  overrunning  budgeted  cost.  Early  discovery  by  an  SPO 
will  enable  the  enfprcement  of  proper  corrective  action  before  the 
software  problems  lead  to  large  cost  overruns  and  schedule  delays 
in  the  overall  system  acquisition. 


i 
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GRAPHIC  ANALYSIS 


CUMUlATIVt  NBVOftMAMCC  CUMSNT  VANANCt 


The  graphic  analysis  of  financial  data  associated  with  a  CWBS 
element  included  in  the  problem  list,  can  be  made  to  note  cost  and 
schedule  element  trends.  The  illustration  shows  two  graphic 
representations  of  cost  and  schedule  financial  data  commonly  used 
with  data  collected  under  the  standard  cost  performance  report. 

These  graphs  also  apply  to  the  software  cost  performance  report. 

The  Cumulative  Performance  graph  plots  BCWS,  SCWP,  and  ACWP 
against  time  on  a  cumulative  basis  from  the  beginning  of  the 
contract.  The  example  shown  by  the  illustration  indicates  a 
software  activity  behind  schedule  anu  overrunning  budgeted  cost, 
with  both  cost  and  schedule  trends  progressively  worsening.  This 
presentation  smooths  the  data  to  a  certain  extent  to  Illustrate  the 
performance  trend,  but  does  not  effectively  show  short  term 
performance  (e.g.,  over  the  last  two  months).  The  current  variance 
graph,  hich  plots  the  monthly  variances  against  time. does  a  good  Job 
depicting  short-term  performance.  These  graphs  can  be.  generated  by 
Government  software. 
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Project  Historical  File  Technical  Printouts 


PROJECT 
HISTORICAL 
FILE 


0 

© 

i 

i 

Historical  file  data  can  be  used  to  explain  software  problems, 
so  that  proper  corrective  action  can  be  initiated.  The  most  obvious 
explanation  for  cost  overruns  and  schedule  delays  would  be  a 
software  CPCI  acquisition  change  in  scope.  A  historical  file 
printout  lists  changes  and  deviations  associated  with  a  CPCI, 
including  any  requirements  changes,  design  changes,  and  engineering 
change  proposals. 

A  second  obvious  explanation  for  cost/schedule  anomalies  would 
be  significant  growth  in  the  s: ze  of  a  CPCI  over  its  development 
life  cycle,  as  shown  by  a  historical  file  printout.  If  current 
estimates  are  significantly  greater  than  the  origi  :  .  ...  dilates  of 

4mplementation  code  required,  then  size  growth  could  well  account 
..'or  poor  financial  data. 

A  historical  file  printout  of  technical  data  itself  could  flag 
potential  software  problems.  For  example,  a  significant  change  in 
the  number  of  personnel  assigned  to  a  CPCI  could  indicate  a  problem. 
Such  a  change  in  staff  size  couj.d  indicate  that  a  contractor  has 
recognized  a  development  problem. 


Questions  —  Pilot  Review 


The  cost  for  a  contractor  to  report  the  software  cost 
performance  data  has  been  estimated  to  be  about  1  to  2  percent  of 
the  software  acquisition  cost.  These  estimates  were  made  by 
carefully  considering  contractor  tasks  required  to  report  data,  but 
are  unproven.  Cost  and  other  questions  of  the  SARE  reporting 
methodology  have  been  raised  by  reviewers,  who  have  asked  the 
following: 

1.  Does  the  methodology  encroach  on  a  contractor's  method  of 
doing  business? 

2.  Will  the  contractor's  cost  to  update  or  modify  his 
accounting  system  be  significant? 

3.  Are  the  CWBS  breakouts  artificial,  causing  inaccurate 
reporting  of  software  costs? 

A.  Is  the  S/W  CPR  data  too  detailed  for  program  office  use? 

5.  Is  automated  report  generation  helpful  or  unnecessary? 

We  are  aware  of  these  and  other  issues  and  believe  a  pilot 
application  of  the  SARE  reporting  methodology  should  answer  many  of 
the  questions,  A  pilot  application  is  planned  to  start  later  this 
year. 
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Multi  Project  Data  Base  — 
Future  Software  Data  Base 
Management  System 


The  methodology  will  be  reviewed  after  the  pilot  with  respect 
to  implementation  costs,  data  collection  proficiency,  and  use  as  a 
monitoring  tool.  Assuming  a  positive  evaluation  and  efficient 

revision,  the  ESD  intends  to  apply  the  methodology  to  several 
acquisitions.  In  addition,  the  ESD  will  incorporate  resulting 
project  historical  file  data  into  a  multiproject  ESD  software  data 
base,  using  the  most  appropriate  commercially  available  data  base 
management  system.  This  data  will  be  used  to  formulate,  calibrate, 
and  validate  software  cost  estimation  methods. 

Presently,  several  cost  estimation  models  exist  (e.g.,  PRICE  S, 
SLIM).  Also,  software  acquisition  models  are  under  development  for 
use  in  cost  and  schedule  estimation.  Although  the  models  vary 
greatly  in  how  they  approach  the  software  cost  estimation  problem, 
they  all  have  one  t^ing  in  common;  the  confidence  of  their 
predictions  is  limited  without  access  to  a  well-structured, 
comprehensive  software  data  base  used  for  calibration  and  validation 
nurposes.  The  multiproject  data  base  would  fill  this  missing  link 
in  the  estimation  process. 
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